Conversation
There was a problem hiding this comment.
Code Review
This pull request enables the mypy type checker in the noxfile.py for the google-cloud-ndb package, which was previously skipped. The feedback suggests improving the mypy command to correctly handle positional arguments and adding the --namespace-packages flag to ensure proper module resolution within the google namespace.
| yield self._next_batch() # First time | ||
|
|
||
| assert self._batch is not None | ||
| assert self._index is not None |
There was a problem hiding this comment.
Did you mean to have assert as part of this code path? Usually this is only used in tests
There was a problem hiding this comment.
I looked for ways to provide type narrowing for mypy and assert is one way it can be done.
It is admittedly controversial for some in production code.
I went ahead and switched the asserts into if clauses throughout.
| if isinstance(value, str): | ||
| try: | ||
| time_tuple = time.strptime(value, "%H:%M:%S") | ||
| st = time.strptime(value, "%H:%M:%S") |
There was a problem hiding this comment.
Replace st with a more meaningful name
There was a problem hiding this comment.
Gemini suggested this refactor to clean up this code and reduce the number of if statements
import datetime
import time
def _time_function(values):
# 1. Normalize input into a tuple of integers
if len(values) == 1:
val = values[0]
if isinstance(val, str):
try:
st = time.strptime(val, "%H:%M:%S")
t_tuple = (st.tm_hour, st.tm_min, st.tm_sec)
except ValueError as e:
_raise_cast_error(f"Format error: {e}, {values}")
elif isinstance(val, int):
t_tuple = (val,)
else:
_raise_cast_error(f"Invalid type {type(val)} for time()")
elif 1 < len(values) < 4:
t_tuple = tuple(values)
else:
_raise_cast_error(f"Invalid number of arguments: {len(values)}")
# 2. Convert tuple to datetime.time using unpacking
try:
return datetime.time(*t_tuple)
except (ValueError, TypeError) as e:
_raise_cast_error(f"Time conversion error: {e}, {values}")
There was a problem hiding this comment.
These two code blocks are fundamentally the same with the exception of item # 2. Convert tuple to datetime.time using unpacking so my comments focus on that.
In your comment we assert that Gemini recommends this refactor to reduce the number of if statements.
Funnily enough:
Those if statements were also recommended by Gemini to enable greater type safety with mypy, because the datetime.time(*t_tuple) expression caused mypy errors.
google/cloud/ndb/_gql.py:793: error: Too many arguments for "time" [call-arg]
google/cloud/ndb/_gql.py:793: error: Too many positional arguments for "time" [misc]
Here is Gemini's rationale for the approach taken in this PR (this was the third change we made and Gemini felt it was the most important):
< BEGINNING of GEMINI's exposition >
- Exploding the "Splat" Operator (*t_tuple)
This is the most important change for Mypy:
Old: return datetime.time(*time_tuple)
The Problem: The * (splat) operator tells Python to unpack the tuple into arguments. However, datetime.time() accepts specific arguments: hour, minute, second, microsecond.
If time_tuple has a length of 1, it’s just the hour. If it's length 3, it's H, M, S.
Mypy cannot guarantee at compile-time that the length of the tuple matches what the function expects. It sees *time_tuple and worries you might be passing 10 arguments to a function that takes 4.
New:
if len(t_tuple) == 1:
return datetime.time(t_tuple[0])
elif len(t_tuple) == 2:
return datetime.time(t_tuple[0], t_tuple[1])
# ...The Benefit: This is called Exhaustive Checking or Manual Unrolling.
By checking the len() explicitly, you are "narrowing" the type. Inside the if len(t_tuple) == 1 block, Mypy knows for a fact that t_tuple[0] exists and is the only argument.
This removes all ambiguity. Mypy can now prove the code is safe without needing a # type: ignore.
< END of GEMINI's exposition >
I am happy to resolve this in whatever way feels best to you, but I suspect that if I revert back to using the splat operator, we will have to go through another cycle of trying to find a way to keep mypy happy OR add one or more ignore pragmas.
This PR enables the mypy session in noxfile.py for ndb and aligns it with the GAPIC generator template. Currently, it fails with 196 errors. This is pushed to allow farming out the work to resolve these errors or for separate review.